An Associative Model of Word Use

An Associative Model of Word Use Scott A. Waterman A dissertation submitted to the Department of Computer Science and the Graduate Faculty of Brandeis University in partial fulfillment of the requirements for the degree of Doctor of Philosophy I describe a model of word usage in context that uses the co-occurrence of words with syntactic environments as evidence of the underlying grammatical constraints that license the combinations. Using these associations between words and contexts as a surrogate for their intrinsic grammatical properties enables us to reason about license, restriction, and preference without needing a detailed theory of these phenomena. The syntactic behavior of words can be defined in terms of their licensed occurrence within the space of contexts. Distributional similarity allows us to use observations of word/context co-occurrence to compare the syntactic behavior of words throughout the language. Various measures of structural similarity are introduced to enable us to also compare the intrinsic structural properties of the contexts. By allowing comparisons between similar-but-not-equal contexts, the structural similarity measure increases the breadth of events over which syntactic behavior between words can be compared. Previous studies of distributional similarity are limited by the inability to compare non-equal contexts. This introduction of a measure over the space of contexts also allows us to reduce the sparse nature of lexical co-occurrence data, and to extend the domain of the model to instances not observed during training. This is accomplished by inductively building a class model of the contexts, based on the structural similarity measure. The resulting context classes are shown to be locally coherent, and good predictors of word usage. The reduced representation of the context space then allows us to determine the intrinsic similarity of words, by enabling the comparison of their co-occurrences across the reduced space.